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Abstract 

The practice of medicine is becoming increasingly onerous for doctors, with growing demands coming from both 
patients and peers. Therefore, we need to equip doctors of the future with the appropriate skills to negotiate a very 
complex environment. Assessment should be central to this goal with a renewed focus on a more rounded 
examination process. Rather than a compete reliance on summative assessment, an increased focus should be placed 
on formative assessment, which has been historically neglected in medicine. In addition, it is important for educators 
to realise that a mutimodal approach is required to establish competence in a valid fashion and that each assessment 
modality should be used for both summative and formative assessment where possible. This will not only serve to 
address the external pressures placed on the teacher to streamline students but it will facilitate student learning as 
they become active participants in assessment. 

Other areas of development in medical education going forward include peer assessment, which may be the answer 
to increasing cohort sizes. We also need to address the issue of maintaining adequate standards of assessment. 
Furthermore, we need to conduct further research to ensure that our assessment methods are an adequate predictor of 
future performance otherwise we as educational leaders will be failing both our students and the wider community. 
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1. Introduction 

Assessment is such a pivotal aspect of education as it is a hugely powerful driver of learning for students. To the 
outside world, it represents the quality of students and institutions. Therefore, educators should strive to assess in the 
most beneficial way for students and to meet the changing demands of new generations and of the different 
assessment stakeholders. In this article, the writer would like to critically discuss assessment firstly giving a general 
overview of the topic, with particular focus on the area of anaesthesia. This paper will then look at both assessment 
of learning (AoL) and assessment for learning (AfL), discussing the various benefits and limitations of both 
approaches with particular reference to how these techniques are evolving. Furthermore, the challenges of 
assessment will be explored. Finally, a conclusion from the analysis will be presented focusing on the pertinent 
learning points in relation to the future of medical education. 

2. Overview of Assessment 

Assessment is defined as a process of collecting and evaluating information to measure students’ progress (Brown, 
Bull & Pendlebury,1997, Bloxham & Boyd, 2007). It is well established that assessment shapes the experience of 
students and influences their behaviour more than any other element of their education (Snyder, 1971, Bloxham & 
Boyd, 2007, Epstein, 2007). It is a powerful tool in the armory of the educator and therefore, deserves careful 
consideration. When exploring assessment, there are six key questions which should be addressed; why, what, how, 
when, where and who (Harden & Laidlaw, 2012). 

There are a number of reasons why educators need to assess. Firstly, educators are required to provide a means of 
discriminating between students. Doctors, particularly in the acute area of anaesthesia, must meet minimum safety 
standards. Assessment is also needed to provide institutional quality assurance to key stakeholders. Both of these 
goals are essentially AoL or summative assessment (SA) and have an outward focus serving wider societal aims 
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rather than the student. Placing the student in the centre of the examination process, however, we realise that 
assessment should foster and motivate student learning and equip them with the necessary skills to develop lifelong 
learning (Epstein, 2007, Bennett, 2011, O’Neill, 2014). This is AfL or formative assessment (FA) which serves to 
promote rather than simply measure learning. Each of these aims are equally important but can come into conflict. 
Ideally, we should have a rounded assessment process to address AoL and AfL simultaneously. 

As educators, what blueprint do we use to assess our students so that they acquire the necessary knowledge, skills 
and attitudes? Constructive alignment ensures that the learning objectives are mapped carefully against assessment to 
ensure maximum possible validity (Flamdy, 2006). The four main elements, which need to be aligned, are content, 
intended learning outcomes, pedagogies and assessment principles (Biggs, 1999). When an assessment method lacks 
a comprehensive blueprint, there are a number of deleterious consequences for both educators and students. 
Construct under-representation is a pertinent example of this, which occurs when a topic forming only a small part of 
the curriculum is examined prominently. This not only undermines validity but also can cause huge distress to 
students (Messick, 1989, Coderre, Woloschuk & Mclaughlin, 2009). In order to avoid this, there are many models we 
can use for an assessment blueprint including Miller’s pyramid (1990), Bloom’s taxonomy (1956) and the Dreyfus 
and Dreyfus (1980) spectrum of skills acquisition to name but a few. The common thread to each of these is the 
evolution of student knowledge from the basic information of a novice to the richness of information of an expert 
(Bransford & Donovan, 2005, Schuwirth & van der Vleuten, 2011, Khan & Ramachandran, 2012). 

The assessment methods commonly used in both undergraduate and postgraduate medical education are multiple 
choice questions (MCQ), extended matching questions (EMQ), essay questions, objective structured clinical 
examinations (OSCE) and oral assessment. These are used primarily as SA with a high stakes outcome. Each 
assessment method has a different strength in testing a student’s knowledge, skills or attitudes but we need a 
carefully balanced combination to comprehensively reflect the assessment blueprint (Epstein, 2007, Table 1). 
Furthermore, the same modality can be used for both AoL and AfL and we should be careful not to miss important 
opportunities to fully develop the potential of a given assessment (Flarlen & James, 1997, Epstein, 2007, Harris et al., 
2007). For example, the membership examinations in anaesthesia, usually undertaken after one year of clinical 
experience, consist of two parts; the first involving MCQs and the second involving OSCE stations and an oral 
examination. These are assessed entirely summatively, however, which fails to capitalise on the maximal learning 
potential. By incorporating formative assessment through provision of feedback and stimulation of reflection, trainee 
learning could be exponentially increased. In addition, we also need to look at the ‘one size fits all’ traditional 
approach in how we assess students. An inclusive approach where students and teachers are equal partners in 
choosing how they are assessed has recently been advocated (Francis, 2008, O’Neill, 2010). 
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Table 1. Methods of assessment in medicine 


Method 

Domain 

Type of Use 

Limitation 

Strength 

MCQ 

Knowledge 

Summative 

-Difficult to write 

-Cues 

-High reliability 

-Computer graded 

-Efficient 

SAQs 

Knowledge 

Summative 

Some formative 

Reliability dependant 
on training of graders 

-No cueing 

-Assess problem 
solving 

Essays 

Knowledge 

Summative 

Some formative 

-Time consuming 

-Interrater reliability 

-No cueing 

-Fligher order 
thinking required 

DOPs/MINI-CE 

Skills 

Formative 

-Time consuming 

Feedback by 

X 

Attitude 

Some summative 

-Selective behaviours 

experts 

Oral 

Knowledge 

Attitudes 

Summative 

Some Formative 

-Subjective 

-Time consuming 

-Training of 
examiners 

Feedback by 
experts 

Simulator 

Skills 

Attitudes 

Formative 

Some summative 

-Expensive 

-Can be artificial 

-Tailored to 
educational goals 

-Often reliable 
and credible 

Peer 

Attitudes 

Formative 

-Confidentiality 

-Annonymity 

-Student buy-in 

-Correlates with 
future clinical 
performance 

Self 

Knowledge 

Skills 

Attitudes 

Formative 

-Training required 

-Fosters reflection 
and learning 

Portfolio 

Knowledge 

Skills 

Attitudes 

Formative 

-Time consuming 

-Student selects best 
material 

-Fosters reflection 
and learning 


Modified from Epstein, 2007. 

Where and when to assess are also important considerations. A move towards work placed assessment would 
improve authenticity and help to examine attitudes, which lies at the apex of Miller’s pyramid. Attitudes are very 
difficult in the traditional examination hall (Flin, Patey, Glavin & Maran, 2010, Miller & Archer, 2010). Similarly, 
the philosophy around when to assess is changing, with a move away from solely relying on end of year SA towards 
continuous intervals of examination in order to provide feedback for both the students and teacher. The use of 
portfolios, consisting of reflective commentary, is one method which can be used for continuous assessment which 
can have both SA and FA functions (Challis. 1999, Klenowski, Askew & Carnell, 2006). 

Who should assess anaesthesia trainees is another pertinent question. With the resource limitations that exist in the 
Irish healthcare setting, which is undoubtedly reflected internationally, we should capitalise on the under utilised area 
of peer assessment (PA). It is well established that students learn from receiving feedback from their peers, as well as 
from giving feedback to each other (Ballantyne, Fluges & Mylonas, 2002, Falchikov, 2007, Gielen, Dochy & 
Onghena, 2011, Kearney, 2013). Peer assessment may even stimulate deeper learning than in traditional assessment 
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settings given the intrinsic link between peer assessment and self reflection (Boud, Cohan & Sampson, 1999, Mulder, 
Pearce & Baik, 2014). When introducing peer assessment, we should be aware of the need for annonymity, however, 
which is a central concern of students engaging in this process. 

3. Assessment of learning 

Assessment of learning or summative assessment judges the overall progression of students in a systematic fashion 
(Biggs, 1998, Knight, 2002, Taras, 2005, Harlen & James, 1997). It is a high stakes assessment with a final mark of 
achievement awarded describing the learning achieved against public criteria. SA is a fundamental form of 
assessment as it allows comparisons to be made between students and it facilitates institutional profiling (Harlen & 
James, 1997). Although SA is regarded as not being ‘student centric’ and is often viewed as embodying all the 
negative social aspects of assessment, it allows students to proceed appropriately within the education system 
(Torrance, 1993, Black & Wiliam, 1998, Taras, 2005). 

Summative assessment is the dominant force in both undergraduate and postgraduate medical education. Two of the 
biggest milestones for any anaesthesia trainee are the final medical examinations before graduation from university 
and the membership examinations in anaesthesia, usually undertaken after one year of clincial experience in the 
specialty. A combination of assessment methods are used in both of these examinations including MCQs, EMQs, 
essay questions, OSCEs and oral examinations. Using one author’s experience in anaesthesia, these exams succeed in 
fulfilling certain educational objectives. They are hugely motivating for students due to the importance attached to 
the final grade (Brookhart, 2001). In addition, they are crucial for implementing minimum standards for professional 
practice. AoL fails the student in a number of areas, however, as it does not aim to further student learning or 
enourage students to take responsibility for their own educational needs. It can also create tension between the 
student and teacher (Gipps, 1994). How students cope with the task of passing these examinations is often through a 
combination of rote learning and memorisation which is a very superficial approach. SA encourages students to 
passively accepted ideas without necessarily understanding underlying theory and core foundation principles (Biggs, 
1998, Ramsden, 2003). Furthermore, students often seek out preparatory courses looking for hints about what will be 
examined. Unfortunately, cue seeking behaviour is encouraged by SA (Miller & Parlett, 1974). 

The final medical and anaesthesia membership exams differ in their examination styles. The final medical exams are 
primarily a criterion referenced assessment, grading students against a pre-specified benchmark, where as the 
membership examinations in anaesthesia are a norm referenced assessment, placing trainees onto a bell curve. 
Although norm referenced assessment is unavoidable where limited places are available for progression, it can be a 
very stressful process for students. Norm referenced assessment does not use explicitly described criteria, hence, it 
pits students against each other which places a huge sense of competition on the learning process (Biggs, 1998). To 
counteract this, medical educators should be aware that criterion referenced assessment allows for better constructive 
allignment and maintenance of standards (Cordiner, 2011). 

How students perform in SA is very influential on their educational progress. This is why ensuring appropriate 
robustness of assessment methods used is so important (Harlen & James, 1997, Knight, 2002). Validity refers to 
whether an examination is ‘fit for purpose’ (Cronbach & Meehl, 1955). Reliability, on the other hand, is whether the 
exam is reproducible despite variability of examiners or of the day of examination (Kathirgamanathan & Woods, 
2011, Schuwirth & van der Vleuten, 2011). MCQs are extremely popular in medical education at both an 
undergraduate and postgraduate level. This is unsurprising as MCQs are easy to write, store, administer, provide a 
meaningful assessment of cognitive ability as well as being highly reliable (Haladyna, 1999, Epstein, 2007). 
Educators need to be aware, however, that poorly written MCQs, facilitate success based on the savviness or 
testwiseness of the student rather than their knowledge (Downing, 2002). This contributes to construct-irrelevant 
variance and represents a threat to validity. Both Case and Swanson (1998) and Haladyna and Rodriguez (2013) have 
produced comprehensive guides to MCQ design to increase validity and reliability. As educators of the future, we 
must strive to create the fairest MCQs possible for our students particularly in light of their prominent use in high 
stakes examination. 

To assess derives from the Latin verb ‘assideo, assidere’ meaning ‘to sit beside’. Rather than sitting beside, SA 
conjures up images of the assessor passing judgement by standing over the student. In contrast, FA with it’s 
predominant student focus fits this image better. I will now critically discuss AfL, which is starting to take a more 
central role in assessment of those practising within medicine. 
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4. Assessment for learning 

Research from as far back as thirty years ago has demonstrated that feedback and student engagement are extremely 
powerful tools in stimulating learning (Ramaprasad, 1983, Sadler, 1989). AfL or FA is a process used by educators to 
foster student engagement through the provision of feedback to adjust and improve ongoing teaching and student 
learning (Natriello, 1987, Black & Wiliam, 1998, Heritage, 2010). In contrast to SA, FA focuses on learning rather 
than assessment. Feedback has been shown to raise student achievement but the quality of the feedback is key 
(Shepard, 2009). Appropriate feedback should not only identify the gap between the student’s current and desired 
standards but should be actively used to close it by make changes in teaching practices and engaging the student 
(Archer, 2010). 

Self assessment and PA are additional cornerstones of assessment for learning (Nicol & Macfarland-Dick, 2006). 
Self assessment has been described as a complementary feedback process which encourages students to monitor their 
own learning (Heritage, 2010). Self assessment in the form of both self monitoring and self regulation helps students 
to take responsibility for their learning and develop as self reliant lifelong learners. PA is mutally beneficial for both 
the assessor and the student receiving it (Wiliam & Thompson, 2006). By acting as assessor, a process that involves 
reflection around learning itself, it can further develop a student’s understanding of their own learning. In addition, 
by what a student says about a colleague’s work, we can easily decipher how well they have grasped the underlying 
learning goals and assessment criteria. 

FA is under utilised in medicine, unfortunately to the detriment of students and trainees. As educators, we are now 
meeting a new cohort of students, generation Y, who crave timely feedback and are likely to reject SA as the key 
modality to fulfill their educational needs. In their important paper in 2001, Kathirgamanathan and Woods state that 
the purposes of assessment in anaesthesia are to ensure competence and fitness to practice. No reference is made to 
AfL and this is very much reflected in clinical practice. At the end of each six months rotation in anaesthesia, there is 
a formative assessment of trainee progress. The temptation by tutors, however, is to treat this as a box ticking 
exercise given the time and resource pressures faced within the clinical arena. Furthermore, it is far from ideal to 
allow six months to pass before critiquing trainees as effective feedback should be both timely and specific. All of 
this may be a reflection of the fact that many in the field of medicine lack qualifications in education and are thrust 
into positions as trainors without the appropriate understanding of their role (Ramsde, 2003). 

Simulation training is recognised as a useful tool for clinical and performance based assessment and is widely 
employed as a training technique in anaesthesia (Motola, Devine, Chung, Sullivan & Issenberg, 2013, Gaba, Howard, 
Fish, Smith & Sowb, 2001). At six monthly intervals during anaesthesia training, trainees participate in a mandatory 
simulation session, which is formatively assessed. During these sessions, trainees are paired up and required to 
participate in a scenario which authentically recreates an anaesthetic emergency. These sessions act as powerful 
learning tools to help bridge the gaps in trainee knowledge particularly in relation to the non-technical skills of 
communication, teamwork and leadership. After each scenario, team feedback and debriefing occurs and arguably 
represents the most important aspect of simulation training (McGaghie, Issenberg, Petmsa & Scalese, 2010). Each 
participant along with their peers and the simulation trainers watch a recording of their performance and give 
feedback. Crucially, all events are linked back to the real life clinical environment. Simulation as an assessment 
method is so effective as the scenarios presented reside in the zone of proximal development of trainees. The 
complex resuscitation cases in simulation are often just beyond their comfort zones. By engaging the feedback with 
their trainers and peers, however, the students can further bridge the gaps in their knowledge, skills and attitudes to 
move towards greater clinical competence and independence. 

5. Challenges of assessment 

As medical practitioners, we are constantly required to assess the competence and performance of students, ourselves 
and our colleagues. The area of assessment in medicine is hugely challenging for clinicians and is likely to become 
significantly more demanding going forward. This is largely due to a combination of increasing cohort sizes and 
shrinking budgets (Fottrell, 2006, Gibbs, 2006). 

When assessing students, certain aspects of competence are easier to assess than others. Knowledge can be reliably 
assessed using MCQs, EMQs and essay questions. Similarly, there are a number of effective frameworks for skills 
examination including OSCEs, directly observed procedures (DOP) and mini clinical evaluation exercises 
(MINI-CEX). In contrast attitude, comprising of teamwork, professionalism and communication skills, is more 
difficult to assess (Epstein, 2007). This presents a huge challenge given that quality of patient care is directly linked 
with a clinican’s attitude (CQHCA, 2001). 
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Further challenges exist in relation to the inherent limitations of each individual assessment methods used in 
medicine. For example, MCQs are difficult to write and can result in cueing, essay questions can have high interrater 
reliability, oral examinations are subjective and may have sex and race bias and simulators are expensive and present 
artificial settings. To overcome these challenges, multimodal assessment can be used to provide broader insights into 
trainee competence and allows input from a number of assessors of the student’s performance (Flolmhoe, Sherbina, 
Long, Swing & Frank, 2010). 

Standardisation of assessment is another issue to consider. Doctors are now more mobile than ever and medical 
migration is occuring worldwide, with greater movement towards more affluent countries (Bidwell at al., 2013, Chen 
& Boufford, 2005). Flow do we compare students or trainees from different medical education backgrounds when 
selecting for training posts? Of course, allowing individual medical schools to retain their autonomy regarding the 
examination process facilitates consistency between the curriculum and assessment (Colliver, Vu & Barrows, 1992). 
In the context of Irish medicine, however, huge challenges exist, particularly in relation to intern recruitment. Recent 
changes have created equal access for all students graduating from medical school within the EU for intern jobs in 
Ireland (Bidwell at al., 2013). However, it is unclear whether accreditation standards in relation to curriculum and 
assessment between the participating countries have been formulated. This is a real challenge to medical educators as 
it is well recognised that poor academic quality often results in deficient clinical care despite the underlying 
individual’s level of experience (Hamilton & Pinnegar, 2000). 

Finally, it remains very challenging to correlate assessment and future performance. There is a lack of robust 
evidence linking assessment with safe practice (Epstein, 2007). If this is what we expect of our future physicians, 
then we must strive to address this in order to produce competent doctors and ensure optimal patient care. 

6. Conclusion 

The practice of medicine is becoming increasingly onerous, with growing demands coming from both patients and 
peers. Therefore, we need to equip doctors of the future with the appropriate skills to negotiate a very complex 
environment. Asssement should be central to this goal with increased focus on FA, which has been historically 
neglected in medicine. In addition, it is important for educators to realise that the mutimodal approach is required to 
establish competence in a valid fashion and that each assessment modality should be used for both SA and FA where 
possible. This will not only serve to address the external pressures placed on the teacher to streamline students but it 
will facilitate student learning as they become active participants in assessment. 

Other areas of development going forward include PA, which may be the answer to increasing cohort sizes. We also 
need to address the issue of maintaining adequate standards of assessment. Furthermore, we need to conduct further 
research to ensure that our assessment methods are an adequate predictor of future performance otherwise we as 
educational leaders will be failing both our students and the wider community. As stated by Harden and Laidlaw 
(2012), ‘Students can walk away from bad teaching but are unable to do so with assessment if they are to achieve the 
qualification they seek’. 
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